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Abstract 

Motivated by a recent conjecture concerning the ex- 
pressiveness of declarative networking, we propose 
a formal computation model for "eventually con- 
sistent" distributed querying, based on relational 
transducers. A tight link has been conjectured be- 
tween coordination-freeness of computations, and 
monotonicity of the queries expressed by such com- 
putations. Indeed, we propose a formal definition 
of coordination-freeness and confirm that the class 
of monotone queries is captured by coordination- 
free transducer networks. Coordination-freeness is 
a semantic property, but the syntactic class that we 
define of "oblivious" transducers also captures the 
same class of monotone queries. Transducer net- 
works that are not coordination-free are much more 
powerful. 

1 Introduction 

Declarative networking [17^ is a recent approach 
by which distributed computations and networking 
protocols are modeled and programmed using for- 
malisms based on Datalog. In his keynote speech 
at PODS 2010 [HITn], Hellerstein made a number 
of intriguing conjectures concerning the expressive- 
ness of declarative networking. In the present paper 
we are focusing on the CALM conjecture (Consis- 
tency And Logical Monotonicity). This conjecture 
suggests a strong link between, on the one hand, 
"eventually consistent" and "coordination-free" dis- 
tributed computations, and on the other hand, ex- 
pressibility in monotonic Datalog (Datalog without 
negation or aggregate functions). The conjecture 
was not fully formalized, however; indeed, as Heller- 
stein notes himself, a proper treatment of this con- 
jecture requires crisp definitions of eventual consis- 
tency and coordination, which have been lacking so 
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far. Moreover, it also requires a formal model of 
distributed computation. 

In the present paper, we investigate the CALM 
conjecture in the context of a model for distributed 
database querying. In the model we propose, the 
computation is performed on a network of relational 
transducers. The relational transducer model, in- 
troduced by Abiteboul and Vianu, is well estab- 
lished in database theory research as a model for 
data-centric agents reacting to inputs [51 [TOl [TOl [T^ . 
Relational transducers are firmly grounded in the 
theory of database queries [4j [5] and also have close 
connections with Abstract State Machines [Tn|. It 
thus seems natural to consider networks of rela- 
tional transducers, as we will do here. We give a 
formal operational semantics for such networks, for- 
mally define "eventual consistency" , and formally 
define what it means for a network to compute a 
conventional database query, in order to address the 
expressiveness issues raised by Hellerstein. 

It is less clear, however, how to formalize the in- 
tuitive notion of "coordination". We do not claim 
to resolve this issue definitively, but we propose a 
new, nonobvious definition that appears workable. 
Distributed algorithms requiring coordination are 
viewed as less efficient than coordination-free algo- 
rithms. Hellerstein has identified monotonicity as a 
fundamental property connected with coordination- 
freeness. Indeed, monotonicity enables "embarrass- 
ing parallelism" [TS]: agents working on parts of 
the data in parallel can produce parts of the output 
independently, without the need for coordination. 

One side of the CALM conjecture now states that 
any database query expressible in monotonic Dat- 
alog can be computed in a distributed setting in 
an eventually consistent, coordination-free manner. 
This is the easy side of the conjecture, and in- 
deed we formally confirm it in the following broader 
sense: any monotone query Q can be computed 
by a network of "oblivious" transducers. Oblivious 
transducers are unaware of the network extent (in 
a sense that we will make precise), and every obliv- 



ious transducer network is coordination-free. Here, 
we should note that the transducer model is param- 
eterized by the query language C. that the trans- 
ducer can use to update its local state. Formally, 
the monotone query Q to be computed must be ex- 
pressible in the while-closure of L for the above con- 
firmation to hold. If Q is in Datalog, for example, 
then L can just be the conjunctive queries. 

The other side of the CALM conjecture, that 
the query computed by an eventually consistent, 
coordination-free distributed program is always ex- 
pressible in Datalog, is false when taken literally, as 
we will point out. Nevertheless, we do give a Dat- 
alog version of the conjecture that holds. More im- 
portantly, we confirm the conjecture in the follow- 
ing more general form: coordination-free networks 
of transducers can compute only monotone queries. 
Note that here we are using our newly proposed 
formal definition of coordination-free. 

Finally, the present work also lead us to think 
about the computational power of the language 
Dedalus [5], the Datalog extension used by Heller- 
stein et al. We will show that this language is quite 
powerful by establishing a monotone simulation of 
arbitrary Turing machines. 

This paper is organized as follows. Preliminar- 
ies are in Section O Section [3] introduces net- 
works of transducers. Section S] investigates the use 
of transducer networks for expressing conventional 
database queries in a distributed fashion. Section [5] 
discusses the issue of coordination. Section [5] looks 
into the CALM conjecture. Section [7] presents some 
further results. Section |5] compares our results to 
the language Dedalus. Section IH] is the conclusion. 

In this extended abstract, proofs are mainly given 
on an informal level. 

2 Preliminaries 

We recall some basic notions from the theory of 
database queries [T]. 

A database schema is a finite set S of relation 
names, each with an associated arity (a natural 
number). Assume some infinite universe dom of 
atomic data elements. An instance of a database 
schema S is an assignment / of finite relations on 
dom to the relation names of S, such that when R 
has arity k then I{R) is a A;-ary relation. Equiv- 
alently, we can view an instance as a set of facts 
over iS, where a fact is an expression of the form 
R{ai, . . . ,ak) with ai,...,afe G dom and R G S 



of arity k. The active domain of an instance /, 
denoted by adom(/) , is the set of data elements oc- 
curring in /. 

A k-ary query over 5 is a partial function Q map- 
ping instances of S to k-ary relations on dom such 
that for each I on which Q is defined, the following 
two conditions hold: (i) Q{I) is a k-aiy relation on 
adom(/); and (ii) Q is also defined on the isomor- 
phic instance h{I), for each permutation h of dom, 
and Q{h{I)) — h{Q{I)). A query Q is monotone if 
for any two instances / C J, if Q{I) is defined then 
so is Q(J), and Q{I) C Q{J). 

We assume familiarity with first- order logic (FO) 
as a basic database query language. An FO formula 
^{xi, . . . , Xk) expresses the fc-ary query defined by 
(^(/) = {(ai,...,afc) G adom(/)'= | (adom(/),/) h 
(^[fli, . . . , flfe]}. Note that we evaluate FO formu- 
las on instances under the active-domain semantics. 
The resulting query language is equivalent in ex- 
pressive power to the relational algebra, as well as 
to recursion-free Datalog with negation. 

We will also consider the query languages Dat- 
alog, stratified Datalog (with negation), and while. 
Datalog and stratified Datalog are well known; while 
is the query language obtained from FO by adding 
assignment statements and while-loops. Finally, we 
recall that there exist quite elegant computation- 
ally complete query languages in which every partial 
computable query can be expressed. 

2.1 Relational transducers 

A transducer schema is a tuple (iSin , iSsys , ^msg j 
<5mem) fc) consisting of four disjoint database 
schemas and an output arity k. Here, 'in' stands 
for 'input'; 'sys' stands for 'system'; 'msg' stands 
for 'message'; and 'mem' stands for 'memory'. 

An abstract relational transducer (or just trans- 
ducer for short) over this schema is a collection 
of queries {Q^^ I R ^ S^^z) U {QL I R e 
5mcm} U {Qf^i I R e 5mom} U {Qout}, where 

• every query is over the combined database 
schema 5i„ U S^ys U iSmsg U iSmcm; 

• the arity of each Q^j, each and each Q^^j 
equals the arity of i?; and 

• the arity of Qout equals the output arity k. 

Here, 'snd' stands for 'send'; 'ins' stands for 'in- 
sert'; 'del' stands for 'delete'; and 'out' stands for 
'output'. 
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A state of the transducer is an instance of the 
combined schema Sm U Sgys U iSniom- Intuitively, a 
state just consists of some input relations, some sys- 
tem relations (we will make these precise in the next 
section), and some stored relations that constitute 
the memory of the transducer. 

A message instance is, plainly, an instance of 
iSinsg- Such a message instance can stand for a set 
of messages (facts) received by the transducer, but 
can as well stand for a set of messages sent by the 
transducer. It will always be clear from the context 
which of the two meanings we have. 

Let n be a transducer. A transition of 11 is 
a five-tuple (/, /rev, «/snd, "/out, >/), also denoted as 
I,Iicv '^°"*> J, Jsnd, where / and J are states, I^cv 
and Jsnd are message instances, and Jout is a fc-ary 
relation such that 

• every query of 11 is defined on /' = / U /rev ; 

• J agrees with / on and Sgys] 

• Jsnd(/?), for each R e 5msg, equals Qsndil')i 

• Jout equals (9out(/'); 

• J{R), for each R e iSmem, equals 

{QLii')\QUi')) 

u(gf„,(/')nQii(/')n /(/?)) 

u (/(/?) \(g£,(/')uQii(/'))). 

The intuition behind the instance /' is that 11 sees 
its input, system and memory relations, plus its 
received messages. The transducer does not mod- 
ify the input and system relations. The transducer 
computes new tuples that can be sent out as mes- 
sages; this is the instance Jsnd- The transducer also 
outputs some tuples; this is the relation Jout- These 
outputs cannot later be retracted! Finally the trans- 
ducer updates its memory by inserting and deleting 
some tuples for every memory relation. The intim- 
idating update formula merely expresses that con- 
flicting inserts/ deletes are ignored [111 US]- Note 
that an assignment R := Q can be expressed by 
using Q for and R for Q^^^. 

Note also that transitions are deterministic: if 

/, /rev J, Jsnd and /, /rev J', Jgnd' ttl^n 

*^snd ~ Jsnd; Jout ~ Jout; and J = J. 

An abstract relational transducer as defined 
above is just a collection of queries. If we want 
to write down a transducer then we will of course 
use some query language to express these queries. 



By default, we use FO as the query language. More 
generally, for any query language C we can con- 
sider the language of C-transducers consisting of all 
transducers whose queries are expressed in C. Be- 
cause we are going to place transducers in networks, 
we can think of C as the language that individ- 
ual peers use locally. For example, in the language 
Dedalus [5] , the local language is stratified Datalog. 

3 Transducer networks 

Proviso. From now on we will only consider trans- 
ducers where the system schema Ssys consists of the 
two unary relation names Id and All. 

A network is a finite, connected, undirected graph 
over a set of vertices V C dom. We refer to the 
vertices as nodes. Note that nodes belong to the 
universe dom of atomic data elements; indeed we 
are going to allow that nodes are stored in relations. 
We stress again that a network must be connected. 
This is important to make it possible for fiow of 
information to reach every node. 

A transducer network is a pair (A/", 11) where A/" is 
a network and 11 is a transducer. The general idea 
is that a copy of 11 is running on every node. A 
database instance is distributed over the input re- 
lations of the different nodes. Relation Id contains 
the node identifier where the transducer is running, 
and relation All is the same at all nodes and con- 
tains the set of all nodes. When a node v sends a 
set of facts as messages, these facts are added to 
the message buffers of w's neighbors. Nodes receive 
facts one by one in arbitrary order; messages are 
not necessarily received in the order they have been 
sent. A similar situation can happen in the Internet 
with subsequent TCP/IP connections between the 
same two nodes, where an earlier connection might 
be slower than a later one. Moreover, nodes reg- 
ularly receive a "heartbeat" message which allows 
them to transition even when no message is read. 

We proceed to define the possible runs of a trans- 
ducer network more formally. A configuration of 
the system is a pair 7 — {state, buf) of mappings 
where 

• state maps every node w to a state / = state{v) 
of n, so that I{Id) = {v}, and I{All) = V (the 
set of all nodes of N) . 

• buf maps every node to a finite multiset of facts 
over 5nis(£- 
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Thus, the system relations Id and All give the trans- 
ducer knowledge about the node where it is running 
and about the other nodes in the network. We will 
discuss the use and necessity of these relations ex- 
tensively. 

A general transition of the system is the transfor- 
mation of one configuration to another where some 
node V reads and removes some message instance 
/rev from its input buffer, makes a local transi- 
tion, and sends the resulting message instance Jsnd 
to its neighbors. Formally, a general transition is 
a tuple r = (71, w, /rev, «/out, 72), also denoted as 
71 "^""^ > 72, where 7; — {statei, buf^) for i — 1,2 

is a configuration, w is a node, and /rev Q buf^{v) 
(multiset containment), such that: 

• state2{v') — statei{v') for every node v' 7^ v. 

• There exists Jsnd such that statei(v), Ircv '^°"*> 
state2{v), Jsnd- We call Jout the output of the 
transition and denote it also by out{T). Note 
that, since individual transducer transitions are 
deterministic, Jout and Jsnd are uniquely deter- 
mined by statei{v) and /rev 

• buf2{v) = bufi{v) \ /rev (multiset difference). 

• buf2{v') = buf i{v') for every node v' ^ v that 
is not a neighbor of v. 

• buf2{v') = 6m/]^(u')U Jsnd for every node u' that 
is a neighbor of v. Note we are using multiset 
union here. 

We will, however, not use transitions in their most 
general form but only in two special forms: 

Heartbeat transition: is of the form 71 > 79. 

So, some node v transitions without reading 
any message. 

Delivery transition: is of the form 71 "^""^ > 72. 

So, some node v reads a single fact / from its 
received message buffer. 

We only consider these two forms because they ap- 
pear to be the most elementary. We are not sure it 
is realistic to assume that entire message instances 
can be read in one transition. Therefore we limit 
message reading to a single fact. Heartbeat transi- 
tions ensure that nodes can transition even if their 
message buffer is empty. 



For any two configurations 71 and 72 we sim- 
ply write 71 72 to denote that the system can 
transition from 71 to 72 either by some heartbeat 
transition or by some delivery transition. A run 
of the system now is an infinite sequence (r„)„ of 
heartbeat or delivery transitions such that for each 
n, if T„ is 7„ — ?> 7r!+i then t„+i is of the form 
7„+i 7ra+2- In other words, each transition t„ 
with n > works on the result configuration of the 
previous transition. 

The output of a run p is then defined as out{p) — 
|J„ out{Tn)- We note the following: 

Proposition 1. For every run p — (t„)„ there 
exists a natural number m such that out{p) = 
U™=o out{Tn). The number m is called a quiescence 
point for p. 

Indeed, since the initial configuration contains 
only a finite number of distinct atomic data ele- 
ments, and a local query cannot invent new data 
elements, only a finite number of distinct output 
tuples are possible. 

In the language Dedalus [8|, new data elements 
are invented in the form of increasing timestamps, 
so the above proposition does not hold for Dedalus. 
It would be interesting to investigate a version 
of transducer networks with timestamps, or with 
object-creating local queries. 

4 Expressing queries with 
transducer networks 

What does it take for a transducer network to com- 
pute some global query? Here we propose a formal 
definition based on the two properties of consistency 
and network-topology independence. 

An instance / of the input part of the transducer 
schema, iSin, can be distributed over the input rela- 
tions of the nodes on the network. Formally, for any 
instance / of Sim a horizontal partition of I on the 
network A/" is a function H that maps every node v 
to a subset of /, such that / = H{v). The initial 
configuration for // is a configuration (state, buf) 
such that: 

• buf{v) = and state{v){R) = for each node v 
and every R £ iSmom- So, each node starts with 
an empty message buffer and empty memory; 

• For each node v, the restriction of state{v) to 
the input schema Sm equals H(v). 
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A run on H is a run that starts in the initial con- 
figuration for H. 

We also need the notion of fair run. A run is fair 
if every node does a heartbeat transition infinitely 
often, and every fact in every message buffer is even- 
tually taken out by a delivery transition. We omit 
the obvious formalization. 

We now say that a transducer network {J\f, 11) 
is consistent if for every instance / of Sm, all fair 
runs on all possible horizontal partitions of / have 
the same output. Naturally, a consistent transducer 
network is said to compute a query Q over iSin if for 
every instance / of Sin on which Q is defined, every 
fair run on any horizontal partition of / outputs 

Q{i)- 

Example 2. Let us see a simple example of a net- 
work that is not consistent. Consider a network 
with at least two nodes (indeed if the network con- 
sists of a single node the transducer runs all by it- 
self; no messages are delivered and there is only one 
possible run). The input is a set S of data elements. 
Each node sends its part of S to its neighbors. Also, 
each node outputs the first element it receives and 
outputs no further elements. This is easily pro- 
grammed on an FO-transducer. When there are 
at least two nodes and at least two elements in 5*, 
different runs may deliver the elements in different 
orders, so different outputs can be produced, even 
for the same horizontal partition. □ 

Example 3. For a simple example of a consistent 
network, let the input be a binary relation S. Each 
node outputs the identical pairs from its part of 
the input. No messages are sent. This network 
computes the equality selection cr$i=$2('S')- 

An example of a consistent transducer network 
that involves communication, computes the tran- 
sitive closure of in a distributed fashion in the 
well-known way |17j . We present here, a naive, un- 
optimized version. Each node sends its part of the 
input to its neighbors. Each node also sends all 
tuples it receives to its neighbors. In this way the 
input is flooded to all nodes. Each node accumu- 
lates the tuples it receives in a memory relation R. 
Finally, each node maintains a memory relation T 
in which we repeatedly insert SURUTU{ToT) 
(here o stands for relational composition). This re- 
lation T is also output. Thanks to the monotonicity 
of the transitive closure, this transducer network is 
consistent. □ 

We are mainly interested in the case where 
the query can be correctly computed by the dis- 



tributed transducer program, regardless of the net- 
work topology. For example, the transitive closure 
computation from Example 12] is independent of the 
network topology (as long as the network is con- 
nected, but we are requiring that of all networks). 

Formally, a transducer 11 is network-topology in- 
dependent if for every network A/", the system 
(A/", n) is consistent, and regardless of M computes 
the same query Q. We say that Q can be distribut- 
edly computed by 11. 

Example 4. For a simple example of a transducer 
that is consistent for every network topology, but 
that is not network-topology independent, consider 
again as input a set S distributed over the nodes. 
Each node sends its input to its neighbors and also 
sends the elements it receives to its neighbors. Each 
node only outputs the elements it receives. On any 
network with at least two nodes, the identity query 
is computed, but on the network with a single node, 
the empty query is computed. □ 

In order to state a few flrst results in Theorem [Bl 
we introduce the following terminology. 

Oblivious transducer: docs not use the relations 
Id and All. Intuitively, the transducer program 
is unaware of the context in which it is running. 

Inflationary transducer: does not do deletions, 
i.e., each deletion query returns empty on all 
inputs. 

Monotone transducer: uses only monotone local 
queries. 

Lemma 5. 1. There is an inflationary FO- 
transducer such that, on any network, starting 
on any horizontal partition of any instance I of 
iSin, any fair run reaches a configuration where 
every node has a local copy of the entire in- 
stance I in its memory, and an additional flag 
Ready (implemented by a nullary memory re- 
lation) is true. Moreover, the flag Ready does 
not become true at a node before that node has 
the entire instance in its memory. 

2. There is an oblivious, inflationary, monotone 
FO-transducer that accomplishes the same as 
the previous one, except for the flag Ready. 

3. A query is expressible in the language 'while ' 
if and only if it is computable by an FO- 
transducer on a single-node network. 
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Proof. For (1), a multicast protocol 9, is imple- 
mented. Every node v sends out all the facts in its 
local input relations, but each fact is tagged with 
the id of V in an extra coordinate (using relation 
Id). Every node also forwards all input facts it re- 
ceives, and stores received facts in memory. More- 
over, for every input fact received, every node sends 
an acknowledge fact, additionally tagged with its 
own identifier. Every node v keeps a record of for 
which of its local input facts it has received an ack 
from which node. When v has received an ack from 
v' for every local input fact, it sends out a message 
done{v, v') meant for v' . When a node has received 
done from every node (which can be checked using 
the relation All), it knows it is ready. No deletions 
are necessary. 

For (2), the program is much simpler. All nodes 
simply send out their local input facts and forward 
any message they receive. In any fair run, even- 
tually all nodes will have received all input facts. 
Relations Id and All are not needed. 

For (3), on a one-node network there are only 
heartbeat transitions. A while program can be 
simulated by iterated heartbeats using well-known 
techniques [3]. Conversely, it is clear that a one- 
node transducer network can be simulated by a 
while program. The only difficulty is that the trans- 
ducer keeps running indefinitely whereas the while 
program is supposed to stop. Using the technique 
described by Abiteboul and Simon [2J, however, the 
program can detect that it is in an infinite loop. 
This implies that the transducer has repeated a 
state and will output no more new tuples. □ 

Theorem 6. 1. Every query can be distributedly 
computed by some abstract transducer. In par- 
ticular, if C is computationally complete, ev- 
ery partial computable query can be distribut- 
edly computed by an C-transducer. 

2. Every monotone query can be distributedly 
computed by an oblivious, inflationary, mono- 
tone abstract transducer. In particular, if C 
is computationally complete, every partial com- 
putable monotone query can be distributedly 
computed by an oblivious C-transducer. 

3. A query is expressible in the language 'while ' if 
and only if it can be distributedly computed by 
an FO-transducer. 

4. Every monotone query expressible in while can 
be distributedly computed by an oblivious FO- 
transducer. 



5. A query is in Datalog if and only if it can be 
distributedly computed by an oblivious, infla- 
tionary, nonrecursive-Datalog-transducer. 

Proof. For ([T]), to distributedly compute any query 
Q, we first run the transducer from Lemma [SJl) to 
obtain the entire input instance. Then we apply 
and output Q. 

For ([2]), the idea is the same, but we now use 
the transducer from Lemma[5j2). We continuously 
apply Q to the part of the input instance already re- 
ceived, and output the result. Since Q is monotone, 
no incorrect tuples are output. 

For ([2]) we only still have to argue the only- 
if implication. We first run the transducer from 
Lemma [UJl) to obtain the entire input instance. 
Then every node can act as if it is on its own, ignor- 
ing any remaining incoming messages and simulate 
the while-pTOgTSLm. 

For (|4]) the idea is the same as in ([2]). We re- 
ceive input tuples and store them in memory. We 
continuously recompute the w/iiZe-program, start- 
ing afresh every time a new input fact comes in. 
We use deletion to start afresh. Since the query is 
monotone, no incorrect tuples are output. 

For ([5]) the idea for the only-if implication is again 
the same as in We receive input tuples and 

apply continuously the Tp-operator of the Datalog 
program. By the monotone nature of Datalog eval- 
uation, deletions are not needed, so the transducer 
is infiationary. The Datalog program for the if- 
implication is obtained by taking together the rules 
of all update queries Qf^^ and the output query 
Qout ■ The answer predicate of Qout is the global an- 
swer predicate. The answer predicate of each Qf^^ 
is replaced by i?, so that we obtain a recursive pro- 
gram. □ 

Without proof we note that the transducer from 
Lemma [HKl) can actually be implemented to use 
only unions of conjunctive queries with negation 
(UCQ^). By simulating FO queries by fixed com- 
positions of UCQ^, we obtain: (proof omitted) 

Proposition 7. Every (monotone) query that can 
be distributedly computed by an FO-transducer can 
be distributedly computed by an (oblivious) UCQ-i- 
transducer. 

To conclude, we remark that in a transducer net- 
work of at least two nodes, each node can estab- 
lish a linear order on the active domain, by first 
collecting all input tuples, then sending out all el- 
ements of the active domain, forwarding messages 
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and storing the elements that are received back in 
the order they are received. But such a transducer is 
not truly network-topology independent, as it does 
not work in the same way on a one-node network. 
At any rate, by the well-known characterization of 
PSPACE [P, we obtain: 

Corollary 8. On any network with at least two 
nodes, every PSPACE query can be computed by an 
FO-transducer. 

5 Coordination 

The CALM conjecture hinges on an intuitive notion 
of "coordination-freeness" of certain distributed 
computations. For some tasks, coordination is re- 
quired to reach consistency across the network. 
Two-phase commit is a classical example. The mul- 
ticast protocol used in Lemma OJl) also requires 
heavy coordination. Since coordination typically 
blocks distributed computations, it is good to un- 
derstand precisely when it can be avoided. This is 
what the CALM conjecture is all about. 

It seems hard to give a definitive formalization of 
coordination. Still we offer here a nontrivial defini- 
tion that appears interesting. A very drastic, too 
drastic, definition of coordination-free would be to 
disallow any communication. Our definition is much 
less severe and only requires that the computation 
can succeed without communication on "suitable" 
horizontal partitions. It actually does not matter 
what a suitable partition is, as long as it exists. 
Even under this liberal definition, the CALM con- 
jecture will turn out to hold. 

Formally, consider a network-topology indepen- 
dent transducer 11 and a network A/". We call 11 
coordination-free on N if for every instance / of 5in, 
there exists a horizontal partition H of I on Af and 
a run p of (A/", 11) on i/, in which a quiescence point 
(Proposition [U is already reached by only perform- 
ing heartbeat transitions. Intuitively, if the hori- 
zontal partition is "right" , then no communication 
is required to correctly compute the query. Finally 
we call n coordination-free if it is coordination-free 
on any network. 

Example 9. Consider again the transitive closure 
computation from Example [31 When every node al- 
ready has the full input, they can each individually 
compute the transitive closure. Hence this trans- 
ducer is coordination-free. 

The reader should not be lulled into believing 
that with a coordination-free program it is always 



suflticient to give the full input at all nodes. A (con- 
trived) example of a coordination-free transducer 
that needs communication even if each node has the 
entire input is the following. The input, distributed 
over the nodes as usual, consists of two sets A and 
B, and the query is to determine if at least one of 
A and B are nonempty. If the network has only 
one node (which can be tested by looking at the All 
relation) , the transducer simply outputs the answer 
to the query. Otherwise, it first tests if its local in- 
put fragments A and B are both nonempty. If yes, 
nothing is output, but the value 'true' (encoded by 
the empty tuple) is sent out. Any node that re- 
ceives the message 'true' will output it. When A 
or B is empty locally, the transducer simply out- 
puts the desired output directly. The transducer 
is coordination-free, since if we take care to have at 
least one node that knows A, and another node that 
knows B, but no node knows both, then the query 
can be computed without communication. When A 
and B are both nonempty, however, a run on the 
horizontal partition where every node has the entire 
input will not reach quiescence without communi- 
cation. □ 

Example 10. A simple example of a transducer that 
is not coordination- free, i.e., requires communica- 
tion, is the one that computes the emptiness query 
on an input set S. Since every node can have a 
part of the input, the nodes must coordinate with 
each other to be certain that S is empty at every 
node. Every node sends out its identifier (using 
the relation Id) on condition that its local relation 
S is empty. Received messages are forwarded, so 
that if S is globally empty, eventually all nodes 
will have received the identifiers of all nodes, which 
can be checked using the relation All. When this 
has happened the transducer at each node outputs 
'true'. □ 

Coordination-freeness is undecidable for FO- 
transducers, but we can identify a syntactic class of 
transducers that is guaranteed to be coordination- 
free, and that will prove to have the same expressive 
power as the class of coordination-free transducers. 
Specifically, recall that an oblivious transducer is 
one that does not use the system relations Id and 
All. For now we observe: 

Proposition 11. Every network-topology indepen- 
dent, oblivious transducer is coordination- free. 

Proof. Let 11 be a network-topology independent, 
oblivious transducer. Let Q be the query distribut- 
edly computed by 11. On a one-node network and 



7 



given any input instance /, transducer 11 reaches 
quiescence and outputs Q{I) by doing only heart- 
beat transitions. Now consider any other network, 
any instance / over Sin, and the horizontal parti- 
tion H that places the entire / at every node. Since 
n is oblivious, nodes cannot detect that they are 
on a network with multiple nodes unless they com- 
municate. So, by doing only heartbeat transitions 
initially, every node will act the same as if in a 
one-node network and will already output the entire 
Q(/). □ 

6 The CALM conjecture 

CALM Conjecture ([15 ). A program has an 
eventually consistent, coordination-free execution 
strategy if and only if it is expressible in (mono- 
tonic) Datalog. 

It is not specified what is meant by "program" or 
"strategy" ; here, we will take these terms to mean 
"query" and "distributedly computable by a trans- 
ducer" , respectively. The term "eventually consis- 
tent" is then formalized by our notions of consis- 
tency and network-topology independence. Under 
this interpretation, the conjecture becomes "a query 
can be distributedly computed by a coordination- 
free transducer if and only if it is expressible in 
Datalog" . 

Let us immediately get the if-side of the conjec- 
ture out of the way. It surely holds, and many 
versions of it are already contained in Theorem [B) 
That theorem talks about oblivious transducers, 
but we have seen in Proposition [TT] that these are 
coordination- free. 

As to the only-if side, the explicit mention of Dat- 
alog is a bit of a nuisance. Datalog is limited to 
polynomial time whereas there certainly are mono- 
tone queries outside PTIME. This continues to hold 
for queries expressible in the language Dedalus that 
Hellerstcin uses; we will show this in Section [51 We 
also mention the celebrated paper [7 where Afrati, 
Cosmadakis and Yannakakis show that even within 
PTIME there exist queries that are monotone but 
not expressible in Datalog. 

We will see in Corollarv [Til( 5|) how a Datalog ver- 
sion of the CALM conjecture can be obtained. Dat- 
alog aside, however, the true emphasis of the CALM 
conjecture clearly lies in the monotonicity aspect. 
Indeed we confirm it in this sense: 



Theorem 12. Every query that is distributedly 
computed by a coordination-free transducer is mono- 
tone. 

Proof. Let Q be the query distributedly computed 
by the coordination-free transducer 11. Let / C J be 
two input instances and let t G Q{I). We must show 
t S Q{J). Consider a network N with at least two 
nodes. Since 11 is coordination-free, there exists a 
horizontal partition H oil such that Q{I) is already 
output distributedly over the nodes, by letting the 
nodes do only heartbeat transitions. Let ?; be a node 
where t is output. Let v' be a node different from v 
and consider the horizontal partition H' of J where 
H'{v) = H{v) and H'{v') = H{v') U ( J \ /). The 
partial run on H where v first does only heartbeat 
transitions until t is output, is also a partial run on 
H' . This partial run can be extended to a fair run, 
so t is output by some fair run of (A/", 11) on H' . 
Since (A/", 11) is consistent, t will also be output in 
any other fair run on any horizontal partition of J. 
Hence, t belongs to the query computed by (TV, 11) 
applied to J. Moreover, 11 is network-topology in- 
dependent, so t belongs to Q{J). □ 

Corollary 13 (CALM Property). The following 
are equivalent for any query Q: 

1. Q can be distributedly computed by a transducer 
that is coordination-free. 

2. Q can be distributedly computed by a transducer 
that is oblivious. 

3. Q is monotone. 

Proof. Theorem ini yields (3) (2); Proposition [TT] 
yields (2) ^ (1); Theorem [TU yields (1) ^ (3). □ 

Similarly we obtain the following versions of the 
CALM property: 

Corollary 14. The following groups of statements 
are equivalent for any query Q: 

1. Let C be a computationally complete query lan- 
guage. 

(a) Q can be distributedly computed by a 
coordination-free C-transducer. 

(b) Q can be distributedly computed by an 
oblivious C-transducer. 

(c) Q is monotone and partial computable. 

2. (a) Q can be distributedly computed by a 

coordination- free FO-transducer. 
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(b) Q can be distributedly computed by an 
oblivious FO-transducer. 

(c) Q is monotone and expressible in the lan- 
guage 'while '. 

3. (a) Q can be distributedly computed by 
a coordination-free nonrecursive-Datalog- 
transducer. 

(b) Q can be distributedly computed by a 
nonrecursive-Datalog-transducer that is 
oblivious. 

(c) Q is expressible in Datalog. 

7 Further results 

It is natural to wonder about variations of our 
model. One question may be about the system re- 
lations Id and All. Without them (the oblivious 
case) we know that we are always coordination-free 
and thus monotone. What if we have only Id or 
only Alll As to coordination-freeness, it is read- 
ily verified that the argument given in the proof of 
Proposition [Til still works in the presence of Id. It 
does not work in the presence of All, and indeed we 
have the following counterexample. 

Example 15. We describe a transducer that is 
network-topology independent, does not use Id, but 
that is not coordination-free. The query expressed 
is simply the identity query on a set S. The trans- 
ducer can detect whether he is alone in the network 
by looking at the relation All. If so, he simply out- 
puts the result. If he is not alone, he sends out a 
ping message. Only upon receiving a ping message 
he outputs the result. Regardless of the horizon- 
tal partition, on a multiple-node network, commu- 
nication is required for the transducer network to 
produce the required output. □ 

So, coordination-freeness is not guaranteed when 
we use the relation All, but yet, monotonicity is not 
lost: 

Theorem 16. Every query distributedly computed 
by a transducer that does not use the system relation 
Id, is monotone. 

Proof. Let 11 be a network-topology independent 
transducer and let Q be the query distributedly 
computed by 11. Let / C J be two input instances 
and let t e Q{I). We must show t G Q{J). Consider 
the network TZa with four nodes 1-2-3-4-1 in the 
form of a ring. Let H be the horizontal partition 



of / that places the entire / at every node. Con- 
sider now the following, fair, run p of (7?.4, 11) on H . 
This particular run has a fifo behavior of the mes- 
sage buffers. We go around the network in rounds. 
The construction is such (proof omitted) that after 
each round, all nodes have the same state and the 
same fifo message buffer queue. In each round, we 
first let each node do a heartbeat transition. Then, 
if some (hence every) input buffer is nonempty, let 
each node do a delivery transition, receiving the first 
tuple in its message buffer. If the buffers are empty, 
we let each node do a second heartbeat transition. 
Since t G Qil), we know that t is output during 
run p. Without loss of generality, assume node 1 
outputs tuple t in round m during run p. 

We now consider the modified network TZ' on the 
same four nodes, obtained by adding the shortcut 
2-4 to TZ4. Consider the horizontal partition H' 
of J defined by if'(l) = H'{2) = H'{4) = I and 
i?'(3) = J \ I. Consider now the following prefix 
p' of a possible run of {TZ',U) on H' . The idea is 
that run p is mimicked until round m, but we ignore 
node 3 completely. The construction is such (proof 
omitted) that after each round, nodes 1, 2 and 4 
have the same state and the same fifo message buffer 
queue as after the same round in p. In each round 
i, we first let each node 1, 2 and 4 do a heartbeat 
transition. Then, if in the same round in p we made 
delivery transitions, then we make the same delivery 
transitions in p' but not for node 3. If in round i 
we did a series of second heartbeat transitions, we 
do the same in p' but again not for node 3. 

The result is that t is also output by node 1 during 
any fair run that has p' as a prefix. Since 11 is 
network-topology independent, we have t G Q{J) 
as desired. □ 

As a corollary we can add two more statements 
to the three equivalent statements of the CALM 
Property (Corollary [T3]): 

Corollary 17. The following are equivalent for any 
query Q: 

1. Q can be distributedly computed by an oblivious 
transducer. 

2. Q can be distributedly computed by a transducer 
that does not use the Id relation. 

3. Q can be distributedly computed by a transducer 
that does not use the All relation. 
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To conclude this section we note that distributed 
algorithms involving a form of coordination typi- 
cally require the participating nodes to have some 
knowledge about the other participating nodes [5]. 
This justifies our modeling of this knowledge in the 
form of the system relations Id and All. Impor- 
tantly, we have shown that these relations are only 
necessary if one wants to compute a nonmonotone 
query in a distributed fashion. 

8 Dedalus 

Dedalus ^ is the declarative language used by 
Hellerstein et al. to model and program network 
protocols. The precise expressive power of Dedalus 
needs to be better understood. Here, we com- 
pare Dedalus to our setting and we also show 
that Dedalus can at least simulate arbitrary Tur- 
ing machines in an eventually consistent manner 
By the time hierarchy theorem [TH], it follows 
that eventually-consistent Dedalus programs are 
not contained in PTIME, let alone in Datalog. 

Dedalus is a temporal version of Datalog with 
negation where the last position of each predicate 
carries a timestamp. All subgoals of any rule must 
be joined on this timestamp. The timestamp of 
the head of the rule can either be the timestamp 
of the body (a "deductive rule" ) , or it can be the 
successor timestamp (an "inductive rule" ) . The de- 
ductive rules must be stratifiable, thus guaranteeing 
modular stratification and a deterministic seman- 
tics through a unique minimal model. Note how 
this corresponds well to transducers using stratified 
Datalog as local query language. 

Furthermore, Dedalus has a non-deterministic 
construct by which facts can be derived with a ran- 
dom timestamp, used to model asynchronous com- 
munication. In our transducer networks, the same 
effect is achieved by the semantics we have given, by 
which one node may send a message in its nth local 
step, whereas another node may receive the message 
in its mth local step where m can be smaller as well 
as larger than n. As long as a Dedalus program is 
monotone in the relations derived by asynchronous 
rules, the program remains deterministic, but there 
is no longer a simple syntactic guarantee for this. 

The feature that makes Dedalus quite powerful 
is that timestamp values can also occur as data 

^This is not the same as saying that all computable 
queries can be expressed in Dedalus; we conjecture this is 
not the case. 



values, i.e., in other predicate positions than the 
last one. This feature, called "entanglement" , is in- 
triguing and makes Dedalus go beyond languages 
such as temporal Datalog [TT]. Note that entangle- 
ment does not involve arithmetic on timestamps; it 
merely allows them to be copied in relations in a 
safe, Datalog-like manner. 

Turing machine simulations in database query 
languages are well known [31 [TJ [TC] , but the Dedalus 
setting is new, so we describe the Turing machine 
simulation in some detail. For any database schema 
<S we can consider the database schema 5*™° with 
the same relation names as S, but in S^™^° each re- 
lation name has arity one higher than in S, in order 
to accommodate timestamps. Dedalus works with 
temporal database instances; these are instances 
over schemas of the form 5*™® in which the last 
coordinate of every fact is a natural number acting 
as timestamp. For any instance / over 5'™° and 
any timestamp value n, let /|„ be the instance over 
S obtained from the facts in / that have timestamp 
n, and let / equal IJ^^ /|„. 

Now let E be an arbitrary but fixed finite alpha- 
bet, and consider the database schema iSs consist- 
ing of relation names Tape of arity two, Begin and 
End of arity one, and a of arity one for each a € E. 
Recall that any string s = ai . . . Op over E can be 
presented as an instance Is over S^,- We consider 
only strings of length at least two. Then Ig con- 
sists of the facts Tape{l,2), Tape{p — l,p), 
Begin{l), End{p), ai(l), ap{p). Such in- 
stances, and isomorphic instances, are known as 
word structures [20^ . 

For any Turing machine M, we define the boolean 
(0-ary) query Qm over the class of temporal in- 
stances over 5^™° as follows. 

• If / is a word structure representing string 
s, and M accepts s, then Qm{I) equals true 
(encoded by the 0-ary relation containing the 
empty tuple). If M does not terminate on s, 
then Qm{I) is undefined. 

• If / contains a word structure, but is not a word 
structure (due to spurious facts), then Qm{I) 
also equals true. 

• In all other cases Qm{I) equals false (encoded 
by the empty 0-ary relation). 

The second item in the definition is there to ensure 
that Qm is monotone; nevertheless, when we give 
Qm a proper word structure as input, a faithful sim- 
ulation of M is required. Hence, the computational 
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complexity of Q m is as high as that of the language 
accepted by M. 

We say that a (deterministic) Dedalus program 
n expresses a boolean query Q over temporal in- 
stances, if for every / such that Q{I) is defined, n(/) 
contains a fact Accept(n) for some n if and only if 
Q{I) is true. Moreover, 11 expresses Q in an even- 
tually consistent way if for every / such that Q{I) 
is defined, there exists n such that n(/)|m = n(/)|„ 
for all m > n. 

Theorem 18. For every Turing machine M , the 
query Qm is expressible in an eventually consistent 
way by a Dedalus program. 

Proof. We only sketch the proof and assume some 
familiarity with Dedalus. The main difficulties to 
overcome are the following. 

1. Detection of a word structure. Since input 
facts can arrive at any timestamp, they are per- 
sisted, e.g., 

a(x, n -t- 1) -s— a(x, n) for each a G E 

(Officially, this should be done using "pos- 
predicates" [8|.) A word structure is detected 
at time n if there is a path in the Tape relation, 
beginning in an element in Begin, and ending 
in an element in End, where all elements on the 
path are labeled, i.e., belong to some a relation. 
This is readily expressed in Datalog. 

2. Detection of spurious tuples. When a word 
structure is already detected, we can detect 
spurious tuples by checking for one of the fol- 
lowing conditions, which can be expressed in 
stratified Datalog: 

(a) Begin and End contain more than a single 
element. 

(b) An element in the active domain is labeled 
by two different alphabet letters. 

(c) Tape is more than a successor relation 
from its begin to its end point, i.e., there 
is an element on the tape with out-degree 
or in-degree more than one, or there is an 
element on the tape that is not reachable 
from Begin. 

(d) There exists a phantom element, i.e., an 
element in the active domain that is not 
labeled, or that is not on the tape. 



3. Turing machine simulation. When a proper 
word structure is discovered, without spurious 
tuples, the simulation of M is started. We copy 
the a predicates to a*^"""' predicates. This is 
necessary because a is persisted, which would 
cause the simulation to be overwritten. And 
we need to continue persisting a because new 
(spurious) a facts may still arrive after the sim- 
ulation has already started. Each transition of 
M goes to the next timestamp. For each state 
q of M we use a predicate q{x,n) that holds 
if M at time n is in state q with its head on 
position X. Timestamp values (entanglement) 
is used to extend the finite tape to the right. 
Care must be taken to do this only when nec- 
essary, to ensure eventual consistency. More- 
over, we must avoid confusing timestamp val- 
ues that may also already occur as input tape 
cells, with timestamp values that are used to 
build the tape extension. Thereto we use a sep- 
arate predicate TapeExt to represent the tape 
extension. For example, the first time that M 
extends the input tape, in some state q and 
seeing letter a at the last input position, is ex- 
pressed by the following rules: 

ExtNext{x,n) ■<— TapeExt{x,y,n) 
TapeExt{x, n, n + 1) ■<— q{x, n), a{x, n), 

End{x, n), -^ExtNext{x, n) 

For positions on the extension tape, we use 
predicates goxt instead of q and a^™"^ instead 
of a"™"'. 

□ 

Distribution is not built in Dedalus and must 
be simulated using data elements serving as loca- 
tion specifiers. The above theorem can be extended 
to a distributed setting where different peers send 
around their input data to their peers. The receiv- 
ing peer treats these messages as EDB facts. This 
works without coordination since the program is 
monotone in the EDB relations. More generally, it 
seems one can define a syntactic class of "oblivious" 
Dedalus programs in analogy to our notion of obliv- 
ious transducers. The restriction would amount to 
disallowing joins on location specifiers. 

9 Conclusion 

Encouraged by Hellerstein [Ml [15] , we have tried in 
this paper to formalize and prove the CALM con- 
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jecture. We do not claim that our approach is the 
only one that works. Yet we believe our approach 
is natural because it is firmly grounded in previous 
database theory practice, and delivers solid results. 

Much further work is possible; we list a few obvi- 
ous topics: 

• Look at HcUcrstein's other conjectures. 

• Investigate the expressiveness of variations or 
extensions of the basic distributed computation 
model presented here. 

• Understand the exact expressive power of the 
Dedalus language, as well as the automated 
verification of Dedalus programs. 

• Identify special cases where essential seman- 
tic notions such as monotonicity, consistency, 
network-topology independence, coordination- 
freeness, are decidable. 
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